Multi-agent artificial intelligence research promises a path to develop intelligent technologies that are more human-like and more human-compatible than those produced by "solipsistic" approaches, which do not consider interactions between agents. Melting Pot is a research tool developed to facilitate work on multi-agent artificial intelligence, and provides an evaluation protocol that measures generalization to novel social partners in a set of canonical test scenarios. Each scenario pairs a physical environment (a "substrate") with a reference set of co-players (a "background population"), to create a social situation with substantial interdependence between the individuals involved. For instance, some scenarios were inspired by institutional-economics-based accounts of natural resource management and public-good-provision dilemmas. Others were inspired by considerations from evolutionary biology, game theory, and artificial life. Melting Pot aims to cover a maximally diverse set of interdependencies and incentives. It includes the commonly-studied extreme cases of perfectly-competitive (zero-sum) motivations and perfectly-cooperative (shared-reward) motivations, but does not stop with them. As in real-life, a clear majority of scenarios in Melting Pot have mixed incentives. They are neither purely competitive nor purely cooperative and thus demand successful agents be able to navigate the resulting ambiguity. Here we describe Melting Pot 2.0, which revises and expands on Melting Pot. We also introduce support for scenarios with asymmetric roles, and explain how to integrate them into the evaluation protocol. This report also contains: (1) details of all substrates and scenarios; (2) a complete description of all baseline algorithms and results. Our intention is for it to serve as a reference for researchers using Melting Pot 2.0.
translated by 谷歌翻译
与人类合作需要迅速适应他们的个人优势,缺点和偏好。遗憾的是,大多数标准的多智能经纪增强学习技术,如自助(SP)或人口剧(PP),产生培训合作伙伴的代理商,并且对人类不完全概括。或者,研究人员可以使用行为克隆收集人体数据,培训人类模型,然后使用该模型培训“人类感知”代理(“行为克隆播放”或BCP)。虽然这种方法可以改善代理商的概括到新的人类共同球员,但它涉及首先收集大量人体数据的繁重和昂贵的步骤。在这里,我们研究如何培训与人类合作伙伴合作的代理的问题,而无需使用人类数据。我们认为这个问题的症结是制作各种培训伙伴。从竞争域中取得成功的多智能经纪人方法绘制灵感,我们发现令人惊讶的简单方法非常有效。我们培养我们的代理商合作伙伴作为对自行发行代理人口的最佳反应及其过去培训的过去检查点,这是我们呼叫虚构共同扮演(FCP)的方法。我们的实验专注于两位运动员协作烹饪模拟器,最近被提议作为与人类协调的挑战问题。我们发现,与新的代理商和人类合作伙伴配对时,FCP代理商会显着高于SP,PP和BCP。此外,人类还报告了强烈的主观偏好,以与所有基线与FCP代理合作。
translated by 谷歌翻译